MACO: A Modality Adversarial and Contrastive Framework for Modality-missing Multi-modal Knowledge Graph Completion

Recent years have seen significant advancements in multi-modal knowledge graph completion (MMKGC). MMKGC enhances knowledge graph completion (KGC) by integrating multi-modal entity information, thereby facilitating the discovery of unobserved triples in the large-scale knowledge graphs (KGs). Nevertheless, existing methods emphasize the design of elegant KGC models to facilitate modality interaction, neglecting the real-life problem of missing modalities in KGs. The missing modality information impedes modal interaction, consequently undermining the model's performance. In this paper, we propose a modality adversarial and contrastive framework (MACO) to solve the modality-missing problem in MMKGC. MACO trains a generator and discriminator adversarially to generate missing modality features that can be incorporated into the MMKGC model. Meanwhile, we design a cross-modal contrastive loss to improve the performance of the generator. Experiments on public benchmarks with further explorations demonstrate that MACO could achieve state-of-the-art results and serve as a versatile framework to bolster various MMKGC models. Our code and benchmark data are available at https://github.com/zjukg/MACO.Comment: This is the ArXiv version of our paper accepted by NLPCC 2023. The code will be released soo

Proof-of-randomness protocol for blockchain consensus: the white paper version 1.0

A proof-of-randomness (PoR) protocol could be a fair and low energy-cost consensus mechanism for blockchains. Each network node of a blockchain could use a true random number generator (TRNG) and hash algorism to fulfil the PoR protocol. In this whitepaper, we give the consensus mechanism of the PoR protocol, and show how it could make the random numbers unforgeable. The PoR protocol could generate a blockchain without any competition of computing power or stake of cryptocurrency. Besides, we give some advantages of integrating quantum random number generator (QRNG) chips in hardware wallets, and also discuss the route to cooperate with quantum key distribution (QKD) technology.Comment: 7 pages, 1 figur

Observation of recoil-induced resonances and electromagnetically induced absorption of cold atoms in diffuse light

In this paper we report an experiment on the observation of the recoil-induced resonances (RIR) and electromagnetically induced absorption (EIA) of cold Rb87 atoms in diffuse light. The pump light of the RIR and the EIA comes from the diffuse light in an integrating sphere, which also serves the cooling light. The probe light beam is a weak laser split from the cooling laser in order to keep the cooling and probe lasers correlated. We measured the RIR and the EIA signal varying with the detuning of the diffuse laser light, and also measured the temperature of the cold atoms at the different detunings. The mechanism of RIR and EIA in the configuration with diffuse-light pumping and laser probing is discussed, and the difference of nonlinear spectra of cold atoms between in diffuse-light cooling system and in optical molasses as well as in a magneto-optical trap (MOT) are studied.Comment: 9 pages, 6 figure

Making Large Language Models Perform Better in Knowledge Graph Completion

Large language model (LLM) based knowledge graph completion (KGC) aims to predict the missing triples in the KGs with LLMs and enrich the KGs to become better web infrastructure, which can benefit a lot of web-based automatic services. However, research about LLM-based KGC is limited and lacks effective utilization of LLM's inference capabilities, which ignores the important structural information in KGs and prevents LLMs from acquiring accurate factual knowledge. In this paper, we discuss how to incorporate the helpful KG structural information into the LLMs, aiming to achieve structrual-aware reasoning in the LLMs. We first transfer the existing LLM paradigms to structural-aware settings and further propose a knowledge prefix adapter (KoPA) to fulfill this stated goal. KoPA employs structural embedding pre-training to capture the structural information of entities and relations in the KG. Then KoPA informs the LLMs of the knowledge prefix adapter which projects the structural embeddings into the textual space and obtains virtual knowledge tokens as a prefix of the input prompt. We conduct comprehensive experiments on these structural-aware LLM-based KGC methods and provide an in-depth analysis comparing how the introduction of structural information would be better for LLM's knowledge reasoning ability. Our code is released at https://github.com/zjukg/KoPA.Comment: Working in progres

Target-oriented Sentiment Classification with Sequential Cross-modal Semantic Graph

Multi-modal aspect-based sentiment classification (MABSC) is task of classifying the sentiment of a target entity mentioned in a sentence and an image. However, previous methods failed to account for the fine-grained semantic association between the image and the text, which resulted in limited identification of fine-grained image aspects and opinions. To address these limitations, in this paper we propose a new approach called SeqCSG, which enhances the encoder-decoder sentiment classification framework using sequential cross-modal semantic graphs. SeqCSG utilizes image captions and scene graphs to extract both global and local fine-grained image information and considers them as elements of the cross-modal semantic graph along with tokens from tweets. The sequential cross-modal semantic graph is represented as a sequence with a multi-modal adjacency matrix indicating relationships between elements. Experimental results show that the approach outperforms existing methods and achieves state-of-the-art performance on two standard datasets. Further analysis has demonstrated that the model can implicitly learn the correlation between fine-grained information of the image and the text with the given target. Our code is available at https://github.com/zjukg/SeqCSG.Comment: ICANN 2023, https://github.com/zjukg/SeqCS